The COBIT Maturity Model in a Vendor Evaluation Case

The maturity model provided by the COBIT Management Guidelines for the 34 COBIT IT processes is becoming an increasingly popular tool to manage the timeless issue of balancing risk and control in a cost-effective manner. Control Objectives for Information and related Technology (COBIT) is published by the IT Governance Institute (ITGI) and Information Systems Audit and Control Foundation (ISACF). The COBIT Maturity Model is an IT governance tool used to measure how well developed the management processes are with respect to internal controls. The maturity model allows an organization to grade itself from nonexistent (0) to optimized (5). Such capability can be exploited by auditors to help management fulfill its IT governance responsibilities, i.e., exercise effective responsibility over the use of IT just like any other part of the business. A fundamental feature of the maturity model is that it allows an organization to measure as-is maturity levels, and define to-be maturity levels as well as gaps to fill. As a result, an organization can discover practical improvements to the system of internal controls of IT. However, maturity levels are not a goal, but rather they are a means to evaluate the adequacy of the internal controls with respect to company business objectives. In volume 6, 2002, of the Information Systems Control Journal, the article “Control and Governance Maturity Survey: Establishing a Reference Benchmark and a Self-assessment Tool,” by Erik Guldentops, CISA, CISM, Wim Van Grembergen, Ph.D., and Steven De Haes, discusses the results of the 2002 ISACA survey on the maturity level of 15 COBIT IT processes. According to the article, survey target processes were selected a year prior by interviewing a group of 20 IT and senior experts. The ISACA survey results can be used as a reference benchmark and a self-assessment tool. The results of the survey cover a broad range of countries, industries and size groups, making them useful for numerous companies worldwide. In an engagement experience, this author participated on a team that used the COBIT Maturity Model to benchmark four possible vendors, and then compared its results to the ISACA survey results. The process undertaken, as well as the lessons learned and the results, is discussed in the remainder of this article. Main Issues and Lessons Learned At the beginning of this benchmarking effort, there were two main issues: • The need for a criterion to choose the processes to benchmark • The need for a method to measure the vendor’s maturity level with respect to the COBIT Maturity Model The processes to benchmark were chosen by scoring the COBIT IT processes on a risk-importance basis, from the point of view of a potential customer. This task followed a logic similar to the one in the risk assessment form of the COBIT Implementation Tool Set. The definition of a method to measure the maturity level required more effort, in part, because the desire was for a method precise and efficient enough to allow for interaction with potential vendors. A questionnaire and a ranking system were developed to compute the maturity level from the questionnaire results. While the approach was not unusual, there were a few new ideas used that proved to be valuable. (These new ideas subsequently have been tested by other AIEA colleagues.) The method used is not strictly incremental and, therefore, does not satisfy the COBIT Maturity Model’s incremental criterion—to check “a posteriori.” However, the method proved to be strong, with respect to the objective of benchmarking the four organizations under examination, and the results were logical given the knowledge collected on the organizations during the benchmarking effort. Moreover, it appears that the method can be further developed to build a strictly incremental approach. Finally, if combined with different methods, the comparison between the benchmarking results and the ISACA survey results provided a basis for an overall discussion on the distribution of the “strongest” and “weakest” areas.